System and method for detecting occluded objects based on image processing

ABSTRACT

The present invention is related to systems and methods for detecting an occluded object based on the shadow of the occluded object. In some examples, a vehicle of the present invention can capture one or more images while operating in an autonomous driving mode, and detecting shadow items within the captured image. In response to detecting a shadow item moving towards the direction of vehicle travel, the vehicle can reduce its speed to avoid a collision, should an occluded object enter the road. The shadow can be detected using image segmentation or a classifier trained using convolutional neural networks or another suitable algorithm, for example.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority benefit of U.S. ProvisionalApplication No. 62/518,524, filed Jun. 12, 2017, the entirety of whichis hereby incorporated by reference.

FIELD OF THE DISCLOSURE

This relates to an autonomous vehicle and, more particularly, to asystem and method of an autonomous vehicle for detecting an occludedobject based on the shadow of the occluded object.

BACKGROUND OF THE DISCLOSURE

Autonomous vehicles, including vehicles operating in a fully autonomousmode, a partially autonomous mode, or a driver assistance mode, candetect objects entering the vehicle's path of travel to avoid acollision. In some examples, however, a pedestrian, animal, or otherobject can suddenly enter the road, giving the vehicle little time toreact. For example, the object can enter the road from behind a parkedvehicle or other large object that conceals the object from one or moresensors (e.g., camera(s), radar, LiDAR, range sensors, ultrasonicsensors) of the autonomous vehicle. In these situations, the vehicle mayhave little time to reduce its speed or come to a complete stop to avoida collision. It is an object of the present invention to use shadowimages to assist with object avoidance during autonomous vehicularnavigation.

SUMMARY OF THE DISCLOSURE

This relates to a system and method of an autonomous vehicle fordetecting an occluded object based on the shadow of the occluded object.In some examples, the vehicle can operate in a shadow detection mode inbased on the vehicle's location. In one embodiment, based on map orlocation data, the vehicle can determine it is currently in apedestrian-heavy zone (e.g., parking lot, city, neighborhood, or schoolzone) and accordingly enter a shadow-detection mode of driving. Whiledriving in the shadow detection mode, the vehicle can capture one ormore images (e.g., still images or videos) with a camera, and identifyone or more shadows of occluded objects moving towards the vehicle'sdirection of travel. The shadows can be detected using imagesegmentation and/or using a classifier trained using convolutionalneural networks or a similar algorithm. In response to detecting ashadow moving towards the vehicle's path of travel, the vehicle canreduce its speed to allow more time to react, should an object enter theroad, for example. In some examples, the shadow of the occluded objectcan be detected even when the occluded object itself may not be detectedby the sensors (e.g., camera(s), LiDAR, radar, ultrasonic sensors, rangesensors).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system block diagram of vehicle controlsystem according to examples of the disclosure.

FIG. 2A illustrates an exemplary vehicle detecting an occluded objectbased on the shadow of the occluded object according to examples of thedisclosure.

FIG. 2B illustrates an exemplary method of detecting an occluded objectduring a fully or partially autonomous driving mode of a vehicleaccording to examples of the disclosure.

FIG. 3A illustrates an exemplary image captured by a camera of vehicleincluding a shadow of an occluded object according to examples of thedisclosure.

FIG. 3B illustrates an exemplary method of identifying a shadow ofoccluded object using image segmentation according to examples of thedisclosure.

FIG. 3C illustrates an exemplary method of identifying a shadow ofoccluded object using a learning algorithm according to examples of thedisclosure.

DETAILED DESCRIPTION

In the following description of examples, references are made to theaccompanying drawings that form a part hereof, and in which it is shownby way of illustration specific examples that can be practiced. It is tobe understood that other examples can be used and structural changes canbe made without departing from the scope of the disclosed examples.Further, in the context of this disclosure, “autonomous driving” (or thelike) can refer to autonomous driving, partially autonomous driving,and/or driver assistance systems.

FIG. 1 illustrates an exemplary system block diagram of vehicle controlsystem 100 according to examples of the disclosure. Vehicle controlsystem 100 can perform any of the methods described below with referenceto FIGS. 2-3. System 100 can be incorporated into a vehicle, such as aconsumer automobile. Vehicle control system 100 can include one or morecameras 106 capable of capturing image data (e.g., video data) of thevehicle's surroundings, as will be described with reference to FIGS.2-3. Vehicle control system 100 can also include one or more othersensors 107 (e.g., radar, ultrasonic, LIDAR, other range sensors, etc.)capable of detecting various characteristics of the vehicle'ssurroundings, and a location system, such as a Global Positioning System(GPS) receiver 108, capable of determining the location of the vehicle.It should be noted that other types of location system can also be used,including cellar, WiFi, or other types of wireless-based locationsystems. Vehicle control system 100 includes an on-board computer 110that is operatively coupled to the cameras 106, sensors 107 and GPSreceiver 108, and that is capable of receiving the image data from thecameras and/or outputs from the sensors 107 and the GPS receiver 108.The on-board computer 110 can also be capable of receiving mapinformation 105 (e.g., via a wireless and/or internet connection at thevehicle). It is understood by ones of ordinary skill in the art that mapdata can be matched to location data in map-matching functions. In someexamples, the vehicle can select an operation mode based on its location(e.g., a parking lot mode, an urban driving mode, a highway mode, oranother location-based operation mode). In accordance with an embodimentof the present invention, in response to determining the vehicle islocation is in a pedestrian heavy-zone where pedestrians, pets, or otherobjects may approach the vehicle's path of travel, the vehicle can entera shadow detection mode as described below. Examples of pedestrian-heavyzones can include parking lots, school zones, neighborhoods, and cities.In accordance with one embodiment of the invention, the on-boardcomputer 110 can be capable of operating in a fully or partiallyautonomous driving mode using camera(s) 106 and GPS receiver 108, asdescribed in this disclosure. In some examples, the on-board computer110 includes storage 112, memory 116, and a processor 114. Processor 114can perform any of the methods described with reference to FIGS. 2-3.Additionally, storage 112 and/or memory 116 can store data andinstructions for performing any of the methods described with referenceto FIGS. 2-3. Storage 112 and/or memory 116 can be any non-transitorycomputer readable storage medium, such as a solid-state drive or a harddisk drive, among other possibilities. The vehicle control system 100can also include a controller 120 capable of controlling one or moreaspects of vehicle operation, such as controlling motion of the vehiclein a fully or partially autonomous driving mode.

In some examples, the vehicle control system 100 can be operativelycoupled to (e.g., via controller 120) one or more actuator systems 130in the vehicle and one or more indicator systems 140 in the vehicle. Theone or more actuator systems 130 can include, but are not limited to, amotor 131 or engine 132, battery system 133, transmission gearing 134,suspension setup 135, brakes 136, steering system 137 and door system138. The vehicle control system 100 can control, via controller 120, oneor more of these actuator systems 130 during vehicle operation; forexample, to open or close one or more of the doors of the vehicle usingthe door actuator system 138, to control the vehicle during fully orpartially autonomous driving operations using the motor 131 or engine132, battery system 133, transmission gearing 134, suspension setup 135,brakes 136 and/or steering system 137, etc. The one or more indicatorsystems 140 can include, but are not limited to, one or more speakers141 in the vehicle (e.g., as part of an entertainment system in thevehicle), one or more lights 142 in the vehicle, one or more displays143 in the vehicle (e.g., as part of a control or entertainment systemin the vehicle) and one or more tactile actuators 144 in the vehicle(e.g., as part of a steering wheel or seat in the vehicle). The vehiclecontrol system 100 can control, via controller 120, one or more of theseindicator systems 140 to provide indications to a driver of the vehicleof one or more aspects of the fully or partially autonomous drivingmode, such as an indication that an occluded object has been detectedbased on detection of the occluded object's shadow.

FIG. 2A illustrates an exemplary vehicle 202 detecting an occludedobject 206 based on the shadow 208 of the occluded object according toexamples of the disclosure. As an example, a vehicle 202 can be drivingin a fully or partially autonomous mode including a shadow detectionmode in a parking lot 200 or other pedestrian-heavy zone (e.g., a schoolzone, a neighborhood, a city). Parking lot 200 can include a pluralityof parked vehicles 204 or other stationary objects that can block anoccluded object 206 from vehicle 202, for example. The occluded objectcan be a pedestrian or animal moving from a position between parked cars204 towards the direction of vehicle 202 travel. In some examples, theoccluded object 206 can be blocked from a camera (e.g., camera(s) 106)or another sensor (e.g., sensor(s) 107) of vehicle 202. For example, theother sensors can include radar, LiDAR, or a range sensor and theoccluded object 206 can essentially be shielded from these sensors bythe parked cars 204. Likewise, the occluded object 206 may not bevisible in one or more images captured by the vehicle camera because itcan be blocked by one or more parked cars 204. Because the occludedobject 204 itself may not be detectable by vehicle 202, a dangeroussituation can arise if the occluded object 204 moves into the path ofvehicle travel with little time for vehicle 202 to react by slowing downor coming to a stop, which can cause a collision.

In some examples, although occluded object 206 may not be detectable byvehicle 202, a shadow 208 of the occluded object can be visible to thevehicle's camera. Based on detecting the shadow 208 in one or morecaptured images and detecting (e.g., using onboard computer 110) thatthe shadow is moving towards the direction of travel of the vehicle 202,the vehicle can reduce its speed or stop to allow extra time to reactshould occluded object 206 enter the vehicle's intended path of travel,for example. In some examples, detecting movement of shadow 208 cancause the vehicle 202 to reduce its speed or stop when the occludedobject 206 is moving towards the vehicle. However, if the shadow 208 isnot moving, which can be indicative of a stationary object such as afire hydrant or parked motorcycle, or is moving away from the directionof vehicle 202 travel, the vehicle may continue to drive withoutreducing its speed or stopping. In some examples, while operating in theshadow detection mode, the vehicle 202 can employ other techniques todetect an occluded object. For example, one or more cameras of thevehicle can capture an image of the occluded object 206 through a windowof a parked car 204 or a radar can detect the occluded object if theradar waves bounce beneath the parked cars 204. Other additionaltechniques of detecting occluded object 206 in conjunction with theshadow-detection mode are possible and multiple techniques can be usedat once to increase the changes of detecting occluded object 206.

FIG. 2B illustrates an exemplary method 250 of detecting an occludedobject during a fully or partially autonomous driving mode of a vehicleaccording to examples of the disclosure. While driving in a fully orpartially autonomous driving mode, the vehicle (e.g., vehicle 202) candetermine its location using GPS 108 and/or map information 105, forexample (step 252 of method 250). In accordance with a determinationthat the vehicle is in a location where pedestrians or other hazards mayenter the vehicle's direction of travel, the vehicle can enter ashadow-detection mode of driving and method 250 can proceed. Forexample, the vehicle can enter the shadow-detection mode and method 250can proceed when the vehicle is in a “pedestrian-heavy zone” such as aparking lot, in a city, in a neighborhood, or in a school zone. Whilethe vehicle is driving, a camera of the vehicle can capture one or moreimages of the vehicle's surroundings (step 254 of method 250). One ormore shadows on the ground in the one or more captured images can bedetected (e.g., by onboard computer 110) (step 256 of method 250).Exemplary details for detecting one or more shadows will be describedwith reference to FIGS. 3A-3C. The vehicle can further determine if thedetected shadows are moving towards the direction of vehicle travel(step 258 of method 250). If the detected shadow is not moving towardsthe direction of vehicle travel (e.g., the shadow is stationary ormoving away from the direction of vehicle travel), method 250 can startover at step 252, for example. If the detected shadow is moving towardsthe direction of vehicle travel, the vehicle can reduce its speed orstop (step 260 of method 250).

FIG. 3A illustrates an exemplary image 300 captured by a camera ofvehicle 202 including a shadow 208 of an occluded object 206 accordingto examples of the disclosure. Image 300 can be captured by one or morecameras (e.g., camera(s) 106) of vehicle 202 and can further includeparked cars 204, shadows 310 of the parked cars, and a horizon 312. Theshadow 208 of the occluded object 206 can be identified using imagesegmentation, as described in further detail with reference to FIG. 3B,and/or using a learning method, as described in further detail withreference to FIG. 3C.

FIG. 3B illustrates an exemplary method 350 of identifying a shadow 208of occluded object 206 using image segmentation according to examples ofthe disclosure. Method 350 can be performed during a shadow detectionmode of the vehicle 202 in accordance with a determination that thevehicle 202 is in a pedestrian-heavy zone, for example. In someexamples, vehicle 202 can identify one or more pixels of image 300 thatcapture the ground (step 352 of method 350). For example, the vehicle202 can identify pixels of image 300 that correspond to objects not onthe ground, such as parked cars 204 and pixels above the horizon 312(e.g., corresponding to the sky, buildings, or traffic lights). In someexamples, vehicle 202 can detect parked cars 204 and any other objectsusing one or more of ultrasonic sensors, radar sensors, LiDAR sensors,and/or range sensors. The detected objects can be associated with one ormore pixels of captured image 300, for example. In some examples, thevehicle 202 can estimate the position of the horizon 312 based on acalibration procedure or a different horizon detection algorithm.Accordingly, by process of elimination, the ground pixels can beidentified.

The vehicle 202 can further segment the ground pixels into regions basedon brightness (step 354 of method 350). For example, pixels proximate toone another having a darkness that is within a threshold difference ofone another can form a segment. Variations in darkness in the image canbe caused by discolorations of the ground, writing or lane markings onthe ground, and/or shadows (e.g., shadow 208 or shadows 310).

In some examples, the vehicle 202 can identify a difference in darkness(black level and/or contrast) of each region compared to the surroundingregions (step 356 of method 350). For example, the shadow 208 of theoccluded object 206 can have a first darkness and one or more regionssurrounding it can have, on average, a second darkness, less than thefirst darkness by at least a threshold difference. The vehicle 202 canidentify one or more “dark” regions surrounded by “light” regions aspossibly corresponding to shadows.

Next, the vehicle 202 can determine whether the dark regions are moving(step 358 of method 350). Detecting which dark regions are moving caneliminate dark regions corresponding to shadows of stationary objects(e.g., shadows 310 of parked cars 204) and dark regions notcorresponding to shadows (e.g., a puddle or another dark spot on theground). In some examples, determining whether the dark regions aremoving can be limited to detecting which dark regions are moving towardsthe path of vehicle 202 travel.

Optionally, vehicle 202 can compare the shape of the dark moving regionsto one or more expected shadow shapes (step 360 of method 350). In someexamples, step 360 can include one or more steps of method 370 describedbelow with reference to FIG. 3C. Vehicle 202 can store (e.g., withinonboard computer 110) one or more reference images corresponding tovarious shadows of people, pets, and other moving objects in a varietyof lighting conditions to use for the comparison, for example.

In some examples, vehicle 202 can identify, using method 350, one ormore shadows 208 of occluded objects 206 that are moving towards thedirection of vehicle travel (step 362 of method 350). In response todetecting one or more shadows of occluded objects moving towards thedirection of vehicle travel, the vehicle 202 can reduce its speed and/orcome to a stop to allow more time to avoid the occluded object, shouldit enter the road.

It should be appreciated that in some embodiments a learning algorithmcan be implemented such as a neural network (deep or shallow, which mayemploy a residual learning framework) and be applied instead of, or inconjunction with, another algorithm described herein to createadditional modes or to improve the above-described modes and/ortransitions between modes. Such learning algorithms may implement afeedforward neural network (e.g., a convolutional neural network) and/ora recurrent neural network, with structured learning, unstructuredlearning, and/or reinforcement learning. In some embodiments,backpropagation may be implemented (e.g., by implementing a supervisedlong short-term memory recurrent neural network, or a max-poolingconvolutional neural network which may run on a graphics processingunit). Moreover, in some embodiments, unstructured learning methods maybe used to improve structured learning methods. Moreover still, in someembodiments, resources such as energy and time may be saved by includingspiking neurons in a neural network (e.g., neurons in a neural networkthat do not fire at each propagation cycle).

FIG. 3C illustrates an exemplary method 370 of identifying a shadow 208of occluded object 206 using a learning algorithm according to examplesof the disclosure. In some examples, method 370 can be performed inaddition or as an alternative to method 350 described above withreference to FIG. 3B. In some examples, one or more steps of method 350can be combined with one or more steps of method 370.

Vehicle 202 can collect example images to form a training data set (step372 of method 370). In some examples, the example images can be capturedby one or more cameras (e.g., camera(s) 106) of vehicle 202.Additionally or alternatively, one or more example images can beuploaded to an onboard computer (e.g., onboard computer 110) of vehicle202 from a different camera. The images can include still images and/orvideos captured in pedestrian-heavy zones such as parking lots, cities,school zones, neighborhoods, and other locations and scenarios where anoccluded object may suddenly enter the path of vehicle travel.

In some examples, the example shadows of moving objects can be segmentedin the example images (step 374 of method 370). Step 374 can includesegmenting the example images manually or using one or more steps ofmethod 350 described above with reference to FIG. 3B to automaticallysegment the images, for example. In some examples, vehicle 202 can store(e.g., using onboard computer 110) the segmented example images.

Next, vehicle 202 can train a classifier to detect shadows of movingobjects (e.g., such as shadow 208 of occluded object 206) using thesegmented example images (step 376 of method 370). In some examples,vehicle 202 can train the classifier using a learning algorithm, such asa Convolutional Neural Network algorithm.

In some examples, steps 372-376 can be part of a vehicle setup procedureperformed at a dealership or factory. Additionally or alternatively,steps 372-376 can be performed multiple times while the vehicle 202 isparked and/or while the vehicle 202 is in use. In some examples, vehicle202 can use a wireless connection to receive one or more segmented orunsegmented example images from a server and/or another vehicle to trainthe classifier to identify shadows of occluded objects (e.g., shadow 208of occluded object 206). The classifier can be trained multiple times oron an ongoing basis as new example images become available to thevehicle 202.

While the vehicle 202 is driving and capturing one or more still orvideo images (e.g., image 300), the classifier can be applied to theimages to identify moving shadows (step 378 of process 370). Forexample, the classifier can associate one or more characteristics of themoving shadows in the training data set with a moving shadow andidentify a moving shadow (step 380 of method 370) in a captured imagebased on identifying one or more of the characteristics in the capturedimage.

In some examples, vehicle 202 can perform method 350 and/or 370 whileoperating in a shadow detection mode. One or more steps of method 350and method 370 can be combined. In some examples, steps of method 350and/or method 370 can be repeated, alternated, performed in any order,and/or skipped.

Thus, examples of the disclosure provide various ways a vehicle candetect an occluded object based on the shadow of the occluded objectwhile driving in an autonomous driving me, allowing the vehicle toreduce its speed to avoid a collision should the object enter thevehicle's path of travel.

Therefore, according to the above, some examples of the disclosure arerelated to a vehicle comprising: one or more cameras; one or moreactuator systems; and a processor operatively coupled to the one or morecameras and the one or more actuator systems, the processor configuredto: identify a shadow in one or more images captured by the one or morecameras; determine whether the shadow is moving in a direction towards adirection of vehicle travel; and in accordance with a determination thatthe shadow is moving in a direction towards the direction of vehicletravel, reducing a speed of the vehicle using the one or more actuatorsystems. Additionally or alternatively, in some examples, the vehiclecomprises a location system and a map interface, wherein the processoris operatively coupled to the location system and the map interface, andthe processor is further configured to: identify a location of thevehicle based on one or more of the location system and the mapinterface; and based on a determination that the vehicle location is ina pedestrian heavy zone, enter a shadow detection mode, wherein theshadow detection mode causes the processor to identify the shadow anddetermine whether the shadow is moving. Additionally or alternatively,in some examples, the shadow is a shadow of an occluded object and theoccluded object is not included in the one or images captured by the oneor more cameras of the vehicle. Additionally or alternatively, in someexamples, the processor is further configured to, in accordance with adetermination that the shadow is stationary or moving in a directionaway from the direction of vehicle travel, maintain the speed of thevehicle using the one or more actuator systems. Additionally oralternatively, in some examples, identifying the shadow in the one ormore images comprises: segmenting a plurality of pixels of the one ormore images into groups based on a darkness of each pixel, whereinpixels within each group have darknesses within a first thresholddifference of each other; and identifying a plurality of dark pixelshaving a first darkness surrounded by a plurality of light pixels havinga second darkness, the first darkness darker than the second darkness byat least a second threshold difference. Additionally or alternatively,in some examples, the vehicle further comprises one or more of a LiDARsensor, an ultrasonic sensor, a radar sensor, and a range sensor,wherein identifying the shadow in the one or more images comprises:identifying a plurality of pixels of the one or more images illustratingan image of a ground based on data from the one or more of the LiDARsensor, the ultrasonic sensor, the radar sensor, and the range sensor;and identifying the shadow within the pixels illustrating the image ofthe ground. Additionally or alternatively, in some examples, identifyingthe shadow in the one or more images comprises comparing the shadow toan expected shadow shape. Additionally or alternatively, in someexamples, identifying the shadow in the one or more images comprises:collecting a plurality of example images; segmenting a plurality ofexample shadows in the plurality of example images; training aclassifier using the plurality of example images; and applying theclassifier to the one or more images.

Some examples of the disclosure are related to a method of operating avehicle in an autonomous driving mode, the method comprising: capturingone or more images at one or more cameras of the vehicle; identifying ashadow in the one or more images; determining whether the shadow ismoving in a direction towards a direction of vehicle travel; and inaccordance with a determination that the shadow is moving in a directiontowards the direction of vehicle travel, reducing a speed of the vehicleusing one or more actuator systems of the vehicle. Additionally oralternatively, in some examples, the method further comprisesidentifying a location of the vehicle based on one or more of a locationsystem and a map interface of the vehicle; and based on a determinationthat the vehicle location is in a pedestrian heavy zone, entering ashadow detection mode, wherein the shadow detection mode causes theprocessor to identify the shadow and determine whether the shadow ismoving. Additionally or alternatively, in some examples, the shadow is ashadow of an occluded object and the occluded object is not included inthe one or images captured by the one or more cameras of the vehicle.Additionally or alternatively, in some examples, the method furthercomprises, in accordance with a determination that the shadow isstationary or moving in a direction away from the direction of vehicletravel, maintaining the speed of the vehicle using the one or moreactuator systems. Additionally or alternatively, in some examples, themethod further comprises segmenting a plurality of pixels of the one ormore images into groups based on a darkness of each pixel, whereinpixels within each group have darknesses within a first thresholddifference of each other; and identifying a plurality of dark pixelshaving a first darkness surrounded by a plurality of light pixels havinga second darkness, the first darkness darker than the second darkness byat least a second threshold difference. Additionally or alternatively,in some examples, the method further comprises identifying a pluralityof pixels of the one or more images illustrating an image of a groundbased on data from one or more of a LiDAR sensor, an ultrasonic sensor,a radar sensor, and a range sensor included in the vehicle; andidentifying the shadow within the pixels illustrating the image of theground. Additionally or alternatively, in some examples, the methodfurther comprises comparing the shadow to an expected shadow shape.Additionally or alternatively, in some examples, the method furthercomprises collecting a plurality of example images; segmenting aplurality of example shadows in the plurality of example images;training a classifier using the plurality of example images; andapplying the classifier to the one or more images.

Although examples of this disclosure have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of examples of this disclosure as defined bythe appended claims.

1. A vehicle comprising: one or more cameras; one or more actuatorsystems; one or more of a LiDAR sensor, an ultrasonic sensor, a radarsensor, and a range sensor; and a processor operatively coupled to theone or more cameras and the one or more actuator systems, the processorconfigured to: identify a shadow in one or more images captured by theone or more cameras; determine whether the shadow is moving in adirection towards a direction of vehicle travel; and in accordance witha determination that the shadow is moving in a direction towards thedirection of vehicle travel, reducing a speed of the vehicle using theone or more actuator systems; wherein identifying a plurality of pixelsof the one or more images illustrating an image of a ground based ondata from the one or more of the LiDAR sensor, the ultrasonic sensor,the radar sensor, and the range sensor; and identifying the shadowwithin the pixels illustrating the image of the ground.
 2. The vehicleof claim 1, further comprising a location system and a map interface,wherein the processor is operatively coupled to the location system andthe map interface, and the processor is further configured to: identifya location of the vehicle based on one or more of the location systemand the map interface; and based on a determination that the vehiclelocation is in a pedestrian heavy zone, enter a shadow detection mode,wherein the shadow detection mode causes the processor to identify theshadow and determine whether the shadow is moving.
 3. The vehicle ofclaim 1, wherein the shadow is a shadow of an occluded object and theoccluded object is not included in the one or images captured by the oneor more cameras of the vehicle.
 4. The vehicle of claim 1, wherein theprocessor is further configured to, in accordance with a determinationthat the shadow is stationary or moving in a direction away from thedirection of vehicle travel, maintain the speed of the vehicle using theone or more actuator systems.
 5. The vehicle of claim 1, whereinidentifying the shadow in the one or more images comprises: segmenting aplurality of pixels of the one or more images into groups based on adarkness of each pixel, wherein pixels within each group have darknesseswithin a first threshold difference of each other; and identifying aplurality of dark pixels having a first darkness surrounded by aplurality of light pixels having a second darkness, the first darknessdarker than the second darkness by at least a second thresholddifference.
 6. The vehicle of claim 1, wherein the radar sensor isconfigured to detect an occluded object when a radar wave from the radarsensor bounces beneath other objects.
 7. The vehicle of claim 1, whereinidentifying the shadow in the one or more images comprises comparing theshadow to an expected shadow shape.
 8. The vehicle of claim 1, whereinidentifying the shadow in the one or more images comprises: collecting aplurality of example images; segmenting a plurality of example shadowsin the plurality of example images; training a classifier using theplurality of example images; and applying the classifier to the one ormore images.
 9. A method of operating a vehicle in an autonomous drivingmode, the method comprising: capturing one or more images at one or morecameras of the vehicle; identifying a plurality of pixels of the one ormore images illustrating an image of a ground based on data from one ormore of a LiDAR sensor, an ultrasonic sensor, a radar sensor, and arange sensor included in the vehicle; identifying the shadow within thepixels illustrating the image of the ground; determining whether theshadow is moving in a direction towards a direction of vehicle travel;and in accordance with a determination that the shadow is moving in adirection towards the direction of vehicle travel, reducing a speed ofthe vehicle using one or more actuator systems of the vehicle.
 10. Themethod of claim 9, further comprising: identifying a location of thevehicle based on one or more of a location system and a map interface ofthe vehicle; and based on a determination that the vehicle location isin a pedestrian heavy zone, entering a shadow detection mode, whereinthe shadow detection mode causes the processor to identify the shadowand determine whether the shadow is moving.
 11. The method of claim 9,wherein the shadow is a shadow of an occluded object and the occludedobject is not included in the one or images captured by the one or morecameras of the vehicle.
 12. The method of claim 9, further comprising:in accordance with a determination that the shadow is stationary ormoving in a direction away from the direction of vehicle travel,maintaining the speed of the vehicle using the one or more actuatorsystems.
 13. The method of claim 9, further comprising: segmenting aplurality of pixels of the one or more images into groups based on adarkness of each pixel, wherein pixels within each group have darknesseswithin a first threshold difference of each other; and identifying aplurality of dark pixels having a first darkness surrounded by aplurality of light pixels having a second darkness, the first darknessdarker than the second darkness by at least a second thresholddifference.
 14. The method of claim 9, further comprising: detecting anoccluded object when a radar wave from the radar sensor bounces beneathother objects.
 15. The method of claim 9, further comprising comparingthe shadow to an expected shadow shape.
 16. The method of claim 9,further comprising: collecting a plurality of example images; segmentinga plurality of example shadows in the plurality of example images;training a classifier using the plurality of example images; andapplying the classifier to the one or more images.
 17. The vehicle ofclaim 1, wherein the processor is further configured to estimate aposition of a horizon and identify ground pixels in response to theestimated position of the horizon.
 18. The vehicle of claim 1, furthercomprising a storage for storing one or more reference imagescorresponding to various shadows of objects in a variety of lightingconditions and the processor is further configured to compare a shape ofthe shadow to one or more stored reference images.
 19. The method ofclaim 9, further comprising: estimating a position of a horizon andidentifying ground pixels in response to the estimated position of thehorizon.
 20. The method of claim 9, further comprising: storing one ormore reference images corresponding to various shadows of objects in avariety of lighting conditions and comparing a shape of the shadow toone or more stored reference images.